Cloud-based medical image processing system with tracking capability

ABSTRACT

A cloud server receives a request for accessing medical image data from a client device, where the cloud server provides image processing services to users in image processing steps, resulting in image views. User privileges of a user are determined for accessing the medical image data. In response to receiving a command having a selection of an image view from the client device, the cloud server provides the medical image data based on the selected one or more image views. The user interactions of the user with the medical image data via the selected image views are tracked, including tracking how long in time the user has spent on a particular image view. The tracked user interactions are stored in a persistent storage, and an analysis is performed on the tracked user interactions stored in the persistent storage to determine an overall usage trend of the image views.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 13/396,548, filed Feb. 14, 2012, which isincorporated by reference herein in its entirety.

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to imageprocessing systems. More particularly, embodiments of the inventionrelate to cloud-based medical image processing systems with trackingcapability.

BACKGROUND

A computerized axial tomography scan (commonly known as a CAT scan or aCT scan) is an x-ray procedure, which combines many x-ray images withthe aid of a computer to generate cross-sectional views of the internalorgans and structures of the body. In each of these views, the bodyimage is seen as an x-ray “slice” of the body. Typically, parallelslices are taken at different levels of the body, i.e., at differentaxial (z-axis) positions. This recorded image is called a tomogram, and“computerized axial tomography” refers to the recorded tomogram“sections” at different axial levels of the body. In multislice CT, atwo-dimensional (2D) array of detector elements replaces the lineararray of detectors used in conventional CT scanners. The 2D detectorarray permits the CT scanner to simultaneously obtain tomographic dataat different slice locations and greatly increases the speed of CT imageacquisition. Multislice CT facilitates a wide range of clinicalapplications, including three-dimensional (3D) imaging, with acapability for scanning large longitudinal volumes with high z-axisresolution.

Magnetic resonance imaging (MRI) is another method of obtaining imagesof the interior of objects, especially the human body. Morespecifically, MRI is a non-invasive, non-x-ray diagnostic techniqueemploying radio-frequency waves and intense magnetic fields to excitemolecules in the object under evaluation. Like a CAT scan, MRI providescomputer-generated image “slices” of the body's internal tissues andorgans. As with CAT scans, MRI facilitates a wide range of clinicalapplications, including 3D imaging, and provides large amounts of databy scanning large volumes with high resolution.

Medical image data, which are collected with medical imaging devices,such as X-ray devices, MRI devices, Ultrasound devices, PositronEmission Tomography (PET) devices or CT devices in the diagnosticimaging departments of medical institutions, are used for an imageinterpretation process called “reading” or “diagnostic reading.” Afteran image interpretation report is generated from the medical image data,the image interpretation report, possibly accompanied by representativeimages or representations of the examination, are sent to the requestingphysicians. Today, these image interpretation reports are usuallydigitized, stored, managed and distributed in plain text in a RadiologyInformation System (RIS) with accompanying representative images and theoriginal examination stored in a Picture Archiving Communication System(PACS) which is often integrated with the RIS.

Typically, prior to the interpretation or reading, medical images may beprocessed or rendered using a variety of imaging processing or renderingtechniques. Recent developments in multi-detector computed tomography(MDCT) scanners and other scanning modalities provide higher spatial andtemporal resolutions than the previous-generation scanners.

Advanced image processing was first performed using computerworkstations. However, there are several limitations to aworkstation-based advanced image processing system. The hardware andsoftware involved with these systems are expensive, and requirecomplicated and time consuming installations. Because the workstationcan only reside in one location, users must physically go to theworkstation to use the advanced image processing software and tools.Also, only one person can use the workstation at a time.

Some have improved on this system by converting the workstation-basedadvanced image processing system to a client-server-based system. Thesesystems offer some improvements over the workstation-based systems inthat a user can use the client remotely, meaning the user does not haveto be physically located near the server, but can use his/her laptop orcomputer elsewhere to use the software and tools for advanced imageprocessing. Also, more than one client can be used with a given serverat one time. This means that more than one user can simultaneously andremotely use the software that is installed on one server. Thecomputational power of the software in a client-server-based system isdistributed between the server and the client. In a “thin client”system, the majority of the computational capabilities exist at theserver. In a “thick client” system, more of the computationalcapabilities, and possibly data, exist on the client.

The hardware software installation and maintenance costs and complexityof a client-server based system are still drawbacks. Also, there can belimitations on the number of simultaneous users that can beaccommodated. Hardware and software must still be installed andmaintained. Generally the information technology (IT) department of thecenter which purchased the system must be heavily involved, which canstrain resources and complicate the installation and maintenanceprocess.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are illustrated by way of example and notlimitation in the figures of the accompanying drawings in which likereferences indicate similar elements.

FIGS. 1A and 1B are block diagrams illustrating a cloud-based imageprocessing system according to certain embodiments of the invention.

FIG. 2 is a block diagram illustrating a cloud-based image processingsystem according to another embodiment of the invention.

FIGS. 3A-3D are diagrams illustrating examples of access control datastructures according to certain embodiments of the invention.

FIGS. 4A-4C are screenshots illustrating certain graphical userinterfaces (GUIs) of a cloud-based image processing system according toone embodiment of the invention.

FIG. 5 is a block diagram illustrating a cloud-based image collaborationsystem according to one embodiment of the invention.

FIGS. 6A-6D are screenshots illustrating certain GUIs of a medical imagecollaboration system according certain embodiments of the invention.

FIG. 7 is a flow diagram illustrating a method for processing medicalimages in a collaboration environment according to one embodiment of theinvention.

FIG. 8 is a flow diagram illustrating a method for processing medicalimages in a collaboration environment according to another embodiment ofthe invention.

FIG. 9 is a block diagram illustrating a cloud-based image processingsystem according to another embodiment of the invention.

FIGS. 10A and 10B are screenshot illustrating examples of graphical userinterfaces for configuring data gateway management according to certainembodiments of the invention.

FIG. 11 is a screenshot illustrating examples of GUIs for configuringanonymous data gateway management according to certain embodiments ofthe invention.

FIGS. 12A-12C are block diagrams illustrating certain systemconfigurations according to some embodiments of the invention.

FIG. 13 is a flow diagram illustrating a method for anonymizing medicaldata according to another embodiment of the invention.

FIG. 14 is a block diagram of a data processing system, which may beused with one embodiment of the invention.

DETAILED DESCRIPTION

Various embodiments and aspects of the inventions will be described withreference to details discussed below, and the accompanying drawings willillustrate the various embodiments. The following description anddrawings are illustrative of the invention and are not to be construedas limiting the invention. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentinvention. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present inventions.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin conjunction with the embodiment can be included in at least oneembodiment of the invention. The appearances of the phrase “in oneembodiment” in various places in the specification do not necessarilyall refer to the same embodiment.

According to some embodiments, advanced image processing systems areprovided as cloud-based systems, particularly, for processing medicalimages. According to one embodiment, a cloud server is configured toprovide advanced image processing services to a variety of clients, suchas physicians from medical institutes, sole practitioners, agents frominsurance companies, patients, medical researchers, regulating bodies,etc. A cloud server, also referred to as an image processing server, hasthe capability of processing one or more medical images to allowmultiple participants to view and process the images eitherindependently or in a collaborated manner or conferencing environment.Different participants may participate in different stages of adiscussion session or a workflow process of the images. Dependent uponthe privileges associated with their roles (e.g., doctors, insuranceagents, patients, or third party data analysts or researchers),different participants may be limited to access only a portion ofinformation relating to the images or a subset of the processing toolswithout compromising the privacy of the patients associated with theimages.

According to some embodiments, a cloud-based medical image processingsystem includes a data gateway manager to automatically and/or manuallytransfer medical data to/from data providers such as medical institutes.Such data gateway management may be performed based on a set of rules orpolicies, which may be configured by an administrator or authorizedpersonnel. In one embodiment, in response to updates to medical imagedata during an image discussion session or image processing operationsperformed at the cloud, the data gateway manager is configured totransmit over a network (e.g., Internet or intranet) the updated imagedata or data representing the difference between the updated image dataand the original image data to a data provider that provided theoriginal medical images. Similarly, the data gateway manager may beconfigured to transfer any new images from the data provider and storethem in a data store of the cloud-based system. In addition, the datagateway manager may further transfer data amongst multiple dataproviders that are associated with the same entity (e.g., multiplefacilities of a medical institute). Furthermore, the cloud-based systemmay automatically perform certain image pre-processing operations of thereceived images using certain advanced image processing resourcesprovided by the cloud systems. The gateway manager may comprise arouter, a computer, software or any combination of these components.

FIGS. 1A and 1B are block diagrams illustrating a cloud-based imageprocessing system according to certain embodiments of the invention.Referring to FIG. 1A, according to one embodiment, system 100 includesone or more entities or institutes 101-102 communicatively coupled tocloud 103 over a network. Entities 101-102 may represent a variety oforganizations such as medical institutes having a variety of facilitiesresiding all over the world. For example, entity 101 may include or beassociated with image capturing device or devices 104, image storagesystem (e.g., PACS) 105, router 106, and/or data gateway manager 107.Image storage system 105 may be maintained by a third party entity thatprovides archiving services to entity 101, which may be accessed byworkstation 108 such as an administrator or user associated with entity101. Note that throughout this application, a medical institute isutilized as an example of an organization entity. However, it is not solimited; other organizations or entities may also be applied.

In one embodiment, cloud 103 may represent a set of servers or clustersof servers associated with a service provider and geographicallydistributed over a network. For example, cloud 103 may be associatedwith a medical image processing service provider such as TeraRecon ofFoster City, Calif. A network may be a local area network (LAN), ametropolitan area network (MAN), a wide area network (WAN) such as theInternet or an intranet, or a combination thereof. Cloud 103 can be madeof a variety of servers and devices capable of providing applicationservices to a variety of clients such as clients 113-116 over a network.In one embodiment, cloud 103 includes one or more cloud servers 109 toprovide image processing services, one or more databases 110 to storeimages and other medical data, and one or more routers 112 to transferdata to/from other entities such as entities 101-102. If the cloudserver consists of a server cluster, or more than one server, rules mayexist which control the transfer of data between the servers in thecluster. For example, there may be reasons why data on a server in onecountry should not be placed on a server in another country.

Server 109 may be an image processing server to provide medical imageprocessing services to clients 113-116 over a network. For example,server 109 may be implemented as part of a TeraRecon AquariusNET™ serverand/or a TeraRecon AquariusAPS server. Data gateway manager 107 and/orrouter 106 may be implemented as part of a TeraRecon AquariusGATEdevice. Medical imaging device 104 may be an image diagnosis device,such as X-ray CT device, MRI scanning device, nuclear medicine device,ultrasound device, or any other medical imaging device. Medical imagingdevice 104 collects information from multiple cross-section views of aspecimen, reconstructs them, and produces medical image data for themultiple cross-section views. Medical imaging device 104 is alsoreferred to as a modality.

Database 110 may be a data store to store medical data such as digitalimaging and communications in medicine (DICOM) compatible data or otherimage data. Database 110 may also incorporate encryption capabilities.Database 110 may include multiple databases and/or may be maintained bya third party vendor such as storage providers. Data store 110 may beimplemented with relational database management systems (RDBMS), e.g.,Oracle™ database or Microsoft® SQL Server, etc. Clients 113-116 mayrepresent a variety of client devices such as a desktop, laptop, tablet,mobile phone, personal digital assistant (PDA), etc. Some of clients113-116 may include a client application (e.g., thin client application)to access resources such as medical image processing tools orapplications hosted by server 109 over a network. Examples of thinclients include a web browser, a phone application and others.

According to one embodiment, server 109 is configured to provideadvanced image processing services to clients 113-116, which mayrepresent physicians from medical institutes, agents from insurancecompanies, patients, medical researchers, etc. Cloud server 109, alsoreferred to as an image processing server, has the capability of hostingone or more medical images and data associated with the medical imagesto allow multiple participants such as clients 113-116, to participatein a discussion/processing forum of the images in a collaborated manneror conferencing environment. Different participants may participate indifferent stages and/or levels of a discussion session or a workflowprocess of the images. Dependent upon the privileges associated withtheir roles (e.g., doctors, insurance agents, patients, or third partydata analysts or researchers), different participants may be limited toaccess only a portion of the information relating to the images or asubset of the tools and functions without compromising the privacy ofthe patients associated with the images.

According to some embodiments, data gateway manager 107 is configured toautomatically or manually transfer medical data to/from data providers(e.g., PACS systems) such as medical institutes. Such data gatewaymanagement may be performed based on a set of rules or policies, whichmay be configured by an administrator or authorized personnel. In oneembodiment, in response to updates of medical images data during animage discussion session or image processing operations performed in thecloud, the data gateway manager is configured to transmit over a network(e.g., Internet) the updated image data or the difference between theupdated image data and the original image data to a data provider suchas PACS 105 that provided the original medical image data. Similarly,data gateway manager 107 can be configured to transmit any new imagesand/or image data from the data provider, where the new images may havebeen captured by an image capturing device such as image capturingdevice 104 associated with entity 101. In addition, data gateway manager107 may further transfer data amongst multiple data providers that isassociated with the same entity (e.g., multiple facilities of a medicalinstitute). Furthermore, cloud 103 may include an advanced preprocessingsystem (not shown) to automatically perform certain pre-processingoperations of the received images using certain advanced imageprocessing resources provided by the cloud systems. In one embodiment,gateway manager 107 is configured to communicate with cloud 103 viacertain Internet ports such as port 80 or 443, etc. The data beingtransferred may be encrypted and/or compressed using a variety ofencryption and compression methods. The term “Internet port” in thiscontext could also be an intranet port, or a private port such as port80 or 443 etc. on an intranet.

Thus, using a cloud-based system for advanced image processing hasseveral advantages. A cloud system refers to a system which isserver-based, and in which the software clients are very thin—possiblyjust a web browser, a web browser with a plug-in, or a mobile or phoneapplication, etc. The server or server cluster in the cloud system isvery powerful computationally and can support several userssimultaneously. The server may reside anywhere and can be managed by athird party so that the users of the software in the cloud system do notneed to concern themselves with software and hardware installation andmaintenance.

A cloud system also allows for dynamic provisioning. For example, iffacility X needs to allow for a peak of 50 users, they currently need a50 user workstation or client-server system. If there are 10 suchfacilities, then a total of 500 users must be provided for withworkstations, or client-server equipment, IT staff, etc. Alternatively,if these same facilities use a cloud service, and for example, theaverage number of simultaneous users at each place is 5 users, then thecloud service only needs to provide enough resource to handle theaverage (5 simultaneous users) plus accommodations for some peaks abovethat. For the 10 facilities, this would mean 50 simultaneous users tocover the average and conservatively 100 simultaneous users to cover thepeaks in usage. This equates to a 150-user system on the cloud systemvs. a 500-user system using workstations or a client-server model,resulting in a 70% saving in cost of equipment and resources etc. Thisallows lower costs and removes the need for the individual sites to haveto manage the asset.

Cloud computing provides computation, software, data access, and storageservices that do not require end-user knowledge of the physical locationand configuration of the system that delivers the services. Cloudcomputing providers deliver applications via the Internet, which areaccessed from Web browsers, desktop and mobile apps, while the businesssoftware and data are stored on servers at a remote location. Cloudapplication services deliver software as a service over the Internet,eliminating the need to install and run the application on thecustomer's own computers and simplifying maintenance and support.

A cloud system can be implemented in a variety of configurations. Forexample, a cloud system can be a public cloud system as shown in FIG.1A, a community cloud system, a hybrid cloud system, a private cloudsystem as shown in FIG. 1B, or a combination thereof. Public clouddescribes cloud computing in the traditional mainstream sense, wherebyresources are dynamically provisioned to the general public on aself-service basis over the Internet, via Web applications/Web services,or other internet services, from an off-site third-party provider whobills on a utility computing basis. Community cloud sharesinfrastructure between several organizations from a specific communitywith common concerns (security, compliance, jurisdiction, etc.), whethermanaged internally or by a third-party and hosted internally orexternally. The costs are spread over fewer users than a public cloud(but more than a private cloud), so only some of the benefits of cloudcomputing are realized. Hybrid cloud is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together, offering the benefits of multiple deployment models.Briefly it can also be defined as a multiple cloud systems which areconnected in a way that allows programs and data to be moved easily fromone deployment system to another. Private cloud is infrastructureoperated solely for a single organization, whether managed internally orby a third-party and hosted internally or externally. Generally, accessto a private cloud is limited to that single organization or itsaffiliates.

With cloud computing, users of clients such as clients 113-116 do nothave to maintain the software and hardware associated with the imageprocessing. The users only need to pay for usage of the resourcesprovided from the cloud as and when they need them, or in a definedarrangement, such as a monthly or annual contract. There is minimal orno setup and users can sign up and use the software immediately. In somesituations, there may be a small software installation, like a Citrix orjava or plug-in. Such a configuration lowers up-front and maintenancecosts for the users and there is no or lower hardware, software, ormaintenance costs. The cloud servers can handle backups and redundanciesand security so the users do not have to worry about these issues. Theusers can have access to all and the newest clinical software withouthaving to install the same. Tools and software are upgraded(automatically or otherwise) at the servers to the latest versions.Access to tools is driven by access level, rather than by softwarelimitations. Cloud servers can have greater computational power topreprocess and process images and they can be larger and more powerfulwith better backup, redundancy, security options. For example, a cloudserver can employ volume rendering techniques available from TeraReconto render large volume of medical images. Further detailed informationconcerning the volume rendering techniques can be found in U.S. Pat.Nos. 6,008,813 and 6,313,841, which are incorporated by referenceherein.

According to one embodiment, image processing services provided by cloud103 can be provided based on a variety of licensing models, such as, forexample, based on the number of users, case uploads (e.g., number ofcases, number of images or volume of image data), case downloads (e.g.,number of cases, number of images or volume of image data), number ofcases processed and/or viewed, image processing requirements, type ofuser (e.g., expert, specialty or general user), by clinical trial or byresearch study, type of case, bandwidth requirements, processingpower/speed requirements, priority to processing power/speed (e.g.,system in ER may pay for higher priority), reimbursement or billing code(e.g., user may only pay to perform certain procedures that arereimbursed by insurance), time using software (e.g., years, months,weeks, days, hours, even minutes), time of day using software, number ofconcurrent users, number of sessions, or any combination thereof.

FIG. 2 is a block diagram illustrating a cloud-based image processingsystem according to another embodiment of the invention. For example,system 200 may be implemented as part of the system as shown in FIGS. 1Aand 1B. Referring to FIG. 2, system 200 includes server 109communicatively coupled to one or more clients 202-203 over network 201,which may be a LAN, MAN, WAN, or a combination thereof. Server 109 isconfigured to provide cloud-based image processing services to clients202-203 based on a variety of usage licensing models. Each of clients202-203 includes a client application such as client applications211-212 to communicate with a server counterpart 209, respectively, toaccess resources provided by server 109. Server application 209 may beimplemented as a virtual server or instance of the server application209, one for each client.

According to one embodiment, server 109 includes, but is not limited to,workflow management system 205, medical data store 206, image processingsystem 207, medical image collaboration system 208, and access controlsystem 210. Medical data store 206 may be implemented as part ofdatabase 110 of FIGS. 1A and 1B. Medical data store 206 is utilized tostore medical images and image data received from medical data center(e.g., PACS systems) 105 or other image storage systems 215 (e.g.,CD-ROMs, or hard drives) and processed by image processing system 207and/or image preprocessing systems 204. Image processing system 207includes a variety of medical imaging processing tools or applicationsthat can be invoked and utilized by clients 202-203 via their respectiveclient applications 211-212, respectively, according to a variety oflicensing terms or agreements. It is possible that in some medicalinstitutes that the image storage system 215 and the image capturingdevice 104 may be combined.

In response to image data received from medical data center 105 or fromimage capturing devices (not shown) or from another image source, suchas a CD or computer desktop, according to one embodiment, imagepreprocessing system 204 may be configured to automatically performcertain preprocesses of the image data and store the preprocessed imagedata in medical data store 206. For example, upon receipt of image datafrom PACS 105 or directly from medical image capturing devices, imagepreprocessing system 204 may automatically perform certain operations,such as bone removal, centerline extraction, sphere finding,registration, parametric map calculation, reformatting, time-densityanalysis, segmentation of structures, and auto-3D operations, and otheroperations. Image preprocessing system 204 may be implemented as aseparate server or alternatively, it may be integrated with server 109.Furthermore, image preprocessing system 204 may perform image datapreprocesses for multiple cloud servers such as server 109.

In one embodiment, a client/server image data processing architecture isinstalled on system 200. The architecture includes client partition(e.g., client applications 211-212) and server partition (e.g., serverapplications 209). The server partition of system 200 runs on the server109, and communicates with its client partition installed on clients202-203, respectively. In one embodiment, server 109 is distributed andrunning on multiple servers. In another embodiment, the system is aWeb-enabled application operating on one or more servers. Any computeror device with Web-browsing application installed may access and utilizethe resources of the system without any, or with minimal, additionalhardware and/or software requirements.

In one embodiment, server 109 may operate as a data server for medicalimage data received from medical image capturing devices. The receivedmedical image data is then stored into medical data store 206. In oneembodiment, for example, when client 202 requests for unprocessedmedical image data, server application 209 retrieves the data from themedical data store 206 and renders the retrieved data on behalf ofclient 202.

Image preprocessing system 204 may further generate workflow informationto be used by workflow management system 205. Workflow management system205 may be a separate server or integrated with server 109. Workflowmanagement system 205 performs multiple functions according to someembodiments of the invention. For example, workflow management system205 performs a data server function in acquiring and storing medicalimage data received from the medical image capturing devices. It mayalso act as a graphic engine or invoke image processing system 207 inprocessing the medical image data to generate 2D or 3D medical imageviews.

In one embodiment, workflow management system 205 invokes imageprocessing system 207 having a graphics engine to perform 2D and 3Dimage generating. When a client (e.g., clients 202-203) requests forcertain medical image views, workflow management system 205 retrievesmedical image data stored in medical data store 206, and renders 2D or3D medical image views from the medical image data. The end results formedical image views are sent to the client.

In one embodiment, when a user making adjustments to the medical imageviews received from server 109, these user adjustment requests are sentback to the workflow management system 205. Workflow management system205 then performs additional graphic processing based on the userrequests, and the newly generated, updated medical image views arereturned to the client. This approach is advantageous because iteliminates the need to transport large quantity of unprocessed medicalimage data across network, while providing 2D or 3D image viewing toclient computers with minimal or no 2D or 3D image processing capacity.

As described above, when implemented as a cloud based application,system 200 includes a client-side partition and a server-side partition.Functionalities of system 200 are distributed to the client-side orserver-side partitions. When a substantial amount of functionalities aredistributed to the client-side partition, system 200 may be referred toas a “thick client” application. Alternatively, when a limited amount offunctionalities are distributed to the client-side partition, while themajority of functionalities are performed by the server-side partition,system 200 may be referred to as a “thin client” application. In anotherembodiment, functionalities of system 200 may be redundantly distributedboth in client-side and server-side partitions. Functionalities mayinclude processing and data. Server 109 may be implemented as a webserver. The web server may be a third-party web server (e.g., Apache™HTTP Server, Microsoft® Internet Information Server and/or Services,etc).

In one embodiment, workflow management system 205 manages the creation,update and deletion of workflow templates. It also performs workflowscene creation when receiving user requests to apply a workflow templateto medical image data. A workflow is defined to capture the repetitivepattern of activities in the process of generating medical image viewsfor diagnosis. A workflow arranges these activities into a process flowaccording to the order of performing each activity. Each of theactivities in the workflow has a clear definition of its functions, theresource required in performing the activity, and the inputs receivedand outputs generated by the activity. Each activity in a workflow isreferred to as a workflow stage, or a workflow element. Withrequirements and responsibilities clearly defined, a workflow stage of aworkflow is designed to perform one specific task in the process ofaccomplishing the goal defined in the workflow. For many medical imagestudies, the patterns of activities to produce medical image views fordiagnosis are usually repetitive and clearly defined. Therefore, it isadvantageous to utilize workflows to model and document real lifemedical image processing practices, ensuring the image processing beingproperly performed under the defined procedural rules of the workflow.The results of the workflow stages can be saved for later review or use.

In one embodiment, a workflow for a specific medical image study ismodeled by a workflow template. A workflow template is a template with apredefined set of workflow stages forming a logical workflow. The orderof processing an activity is modeled by the order established among thepredefined set of workflow stages. In one embodiment, workflow stages ina workflow template are ordered sequentially, with lower order stagesbeing performed before the higher order stages. In another embodiment,dependency relationships are maintained among the workflow stages. Undersuch arrangement, a workflow stage cannot be performed before theworkflow stages it is depending on being performed first. In a furtherembodiment, advanced workflow management allows one workflow stagedepending on multiple workflow stages, or multiple workflow stagesdepending on one workflow stage, etc.

The image processing operations receive medical image data collected bythe medical imaging devices as inputs, process the medical image data,and generate metadata as outputs. Metadata, also known as metadataelements, broadly refers to parameters and/or instructions fordescribing, processing, and/or managing the medical image data. Forinstance, metadata generated by the image processing operations of aworkflow stage includes image processing parameters that can be appliedto medical image data to generate medical image views for diagnosticpurpose. Further, various automatic and manual manipulations of themedical image views can also be captured as metadata. Thus, metadataallows the returning of the system to the state it was in when themetadata was saved.

After a user validates the results generated from processing a workflowstage predefined in the workflow template, workflow management system205 creates a new scene and stores the new scene to the workflow scene.Workflow management system 205 also allows the updating and saving ofscenes during user adjustments of the medical image views generated fromthe scenes. Further detailed information concerning workflow managementsystem 205 can be found in co-pending U.S. patent application Ser. No.12/196,099, entitled “Workflow Template Management for Medical ImageData Processing,” filed Aug. 21, 2008, which is incorporated byreference herein in its entirety.

Referring back to FIG. 2, according to one embodiment, server 109further includes access control system 210 to control access ofresources (e.g., image processing tools) and/or medical data stored inmedical data store 206 from clients 202-203. Clients 202-203 may or maynot access certain portions of resources and/or medicate data stored inmedical data store 206 dependent upon their respective accessprivileges. The access privileges may be determined or configured basedon a set of role-based rules or policies, as shown in FIGS. 3A-3D. Forexample, some users with certain roles can only access some of the toolsprovided by the system as shown in FIG. 3A. Examples of some of thetools available are listed at the end of this document, and includevessel centerline extraction, calcium scoring and others. Some userswith certain roles are limited to some patient information as shown inFIG. 3B. Some users with certain roles can only perform certain steps orstages of the medical image processes as shown in FIG. 3C. Steps orstages are incorporated into the tools (listed at the end of thisdocument) and might include identifying and/or measuring instructions,validation of previously performed steps or stages and others. Someusers with certain roles are limited to certain types of processes asshown in FIG. 3D.

Note that the rules or policies as shown in FIGS. 3A-3D are describedfor the purpose of illustration only; other rules and formats may alsobe applied. According to some embodiments, access levels can beconfigured based on a variety of parameters, such as, for example, typesof tools or steps within a tool, functions (e.g., uploading,downloading, viewing, manipulating, auditing, validating, etc.), abilityto give others access (e.g., second opinion, referrals, experts, family,friend etc.), patients, volume (e.g., may only have access to certainvolume of images/month for example, dependent upon a licensingagreement), medical institution, specialty, reimbursement or billingcode (e.g., may only have access to perform certain procedures that arereimbursed by insurance), admin access level, clinical trial or researchproject, and way of viewing data—some may only be able to see individualpatients, some aggregate data which can be sliced different ways, etc.

FIGS. 4A and 4B are screenshots illustrating certain graphical userinterfaces (GUIs) of a cloud-based image processing system according toone embodiment of the invention. For example, GUI 400 of FIGS. 4A and 4Bmay be presented by workflow management system 205 at clients 202-203 aspart of client applications 211-212 of FIG. 2. Referring to FIGS. 4A and4B, GUI 400 includes one or more controls or buttons 401 to configurecertain settings of the application, such as preferences, help andothers. GUI 400 further includes display area 402 to display a certainpatient study list, where the list may be obtained via a search that isconfigured based on one or more search options 403, such as by patientID, patient name, date, modality or others. GUI 400 further includesdisplay area 404 to display a list of tasks or workflows to be handledby different personnel. The task list may be presented as a table asshown in FIG. 4A or alternatively as a timeline as shown in FIG. 4B. GUI400 further includes image preview area 405 to display a preview of animage of a particular patient in question, optionally including patientinformation 407 and a set of one or more imaging viewing tools 406, suchas brightness, contrast and others. The availability of the patientinformation 407, certain detailed information of image preview 405, andtools 406 may be determined based on access privileges of a particularuser, which may be controlled by access control system 210. Furthermore,GUI 400 further includes a set of one or more file management tools 408for managing image files, such as import, load, send, upload, etc. Notethat GUIs described throughout this application are shown for thepurposes of illustration only; other formats or layouts may also beimplemented. Certain GUI control or button can be activated via avariety of mechanisms, such as keyboard, keypad, touch screen, touchpad, PDA controls, phone controls, mouse click, an interactive voicecommand, or a combination thereof.

Referring back to FIG. 2, according to one embodiment, server 109 mayfurther include a tracking system (not shown), which may be integratedwith server 109 or alternatively maintained by a third party vendor andaccessible by server 109. The tracking system is configured to monitorand track user activities with respect to medical data stored in medicaldata store 206. Because of certain FDA requirements, there is a need totrack what users have accessed the software, when, and the steps theyhave used within the software. There is also a need to track overalltrends in software use, for example how long it takes a user to completea certain type of case, or certain steps, billing trends etc. Accordingto some embodiments, the tracking system is configured to track userswho log in and utilize the software, steps the users perform, date andtime of the accesses, etc. The tracking system can be used to analyzevolumes used on, and performance of, the system. The tracking system canbe used to track FDA/HIPAA compliance. It can also be utilized byinsurance companies to track billing codes and costs. It can be used todetermine trends (time to analyze certain types of cases etc.) Analysisof tracked data can also be used to identify different user types, forexample expert users, casual users, technicians, physicians, etc. Thisinformation can then be used to improve the software product, upsell,improve customer service, improve billing models, etc. The trackingsystem can be used to track aggregate data as well as detailed data.

According to one embodiment, the browser/client/mobile applicationstandards allow easier integration with an electronic health record. Theintegration can be done as seamless as possible, so one does not have toopen separate applications or repeatedly enter login information.Integration may be based on patient ID, or other common parameter whichautomatically links different types of records. It can also be used tolink anonymous cases to online publications—allowing 3D or advancedviews of case images. The cloud-based system is flexible so it can adaptintegration standards as they develop and as they evolve.

As described above, advanced image processing in the cloud model allowsusers from anywhere to access and contribute to the same case or cases.An extension of these concepts is a clinical or research trial. In aclinical or research trial, patient data from different geographicallocations are grouped and tracked over time to identify trends. In thecase of a clinical trial, the trends may be related to a particulartreatment or drug. Advanced image processing is useful in tracking thesetrends. For example, a drug for cancer can be assessed by trackingpatients with cancer who have taken the drug or a placebo. The softwarecan be used to measure the size of the tumor, or other aspects of thepatient, such as side effects. For example, a drug to treat liver cancermay have a potential adverse effect on the function of the heart orother organs. A clinical trial can use advanced image processing totrack not only the health of the liver, but also the health of the heartand other organs over time.

The cloud model allows for doctors or other participants all over theworld to participate. It controls what tools are used and how and bywhom. It allows data to be aggregated because all data is stored on thesame server or cluster of servers. It allows easier training for thedoctors and technicians and others involved in analyzing data forclinical trial or research study. The cloud-based model easily supportsthe role of auditor/quality control person and supports an FDA orcompany oversight role. It monitors data trends as trial is progressingand performs data analysis at the end of a trial/study and also during atrial/study. It also integrates with data analysis software, givingaccess to third parties such as a sponsor or the FDA, and controlsaccess and level of access.

Cloud-Based Medical Data Mining Services

The cloud based system as shown in FIG. 2 allows data from severaldifferent medical centers and geographies to be located on one server orone group of servers. Because the data is all in one place(geographically or virtually), the data can be combined and used in theaggregate. This aggregation can be for one patient, across patients,across time, or any combination of these. According to one embodiment,server 109 further includes a data mining component or system (notshown) configured to perform data mining on medical data received from avariety of data sources. The data mining component may be integratedwith server 109 or alternatively maintained by a third party serviceprovider over a network and invoked by server 109. The data miningsystem is configured to provide cloud-based data mining or analysisservices on medical data stored in medical data store 206 to a varietyof clients over the Internet. The data mining system can perform avariety of mining and analysis operations on demand from a client, orautomatically, and generate and deliver the results to the client basedon a variety of service or licensing agreements.

Some of the reasons one would want to mine aggregated data relating toquantitative image processing include clinical trials, clinicalresearch, trend identification, prediction of disease progress,diagnosis, artificial intelligence, and for use by insurance companies.Quantitative image processing refers to image processing that results ina value such as a measurement of a tumor diameter. According to oneembodiment, the data mining system has the ability to do massive,anonymized, automatic, and continual analysis and trending on all thedata from multiple sources. The data mining system can performquantitative image data analysis that can be performed before a userrequires the data (in the background, at night, etc.) The data miningsystem has the ability to access and use this information quickly and inreal time since the user does not have to download all the images andthen do the analysis every time he/she needs to use the results. It canprovide more flexible licensing, geographies and access, includingcontrolling access by teams, specialties and access levels.

For example, a patient may come into a medical center to have a CT scandone to assess the growth of nodules in his lung. Currently, datarelating to the size of his lung nodule can be collected over time, butit is difficult to put that data into context. Context in this examplecould be either time or population context. Advanced image processingtechniques described herein can be used to measure the location, size,shape and/or volume, or other parameter of the nodules, but there is nota good way to determine the growth rate (time context) or how thispatient compares to other patients with lung nodules (populationcontext). Having access to this information could aide with diagnoses(e.g., whether the nodule is likely cancerous) and treatment (otherpatients with similarly growing lung nodules have responded well tomedication x), among other things.

Over time, the patient image data from several geographically dispersedmedical centers can be stored on a server-based system, data can bemined and analyzed to determine for example: 1) whether an aneurysm islikely to rupture based on certain quantitative characteristics of thataneurysm; 2) whether a tumor is likely to be malignant based on certainquantitative characteristics of the tumor; and 3) whether a growth isgrowing more quickly or more slowly than average and what thatdifference might mean to the patient clinically.

A clinical trial involving a new hip implant can use imaging data todetermine whether the implant is remaining secure over time. Theparticipants in this clinical trial can be geographically dispersedallowing for much more data and therefore a quicker and better studyconclusion. Research around a rare type of brain tumor can advance morequickly because the imaging data can be obtained from any medical centerin the world, thus allowing more of the rare patients to enroll in thestudy. New ways of evaluating aortic aneurysms may be discovered 10years from now and previous imaging data can be re-analyzedretrospectively using the newly discovered information. Or, in thefuture, a doctor may receive a message from the cloud server saying “wehave just developed the ability to detect tumors with more sensitivityand hence please be alerted that we found a possible precursor to atumor in the scan from 5 years ago that patient X had. Follow-uprecommended”. If enough data can be aggregated and analyzed, the systemwill be able to suggest treatments for various diseases which are morelikely to be successful, based on the data analysis. Standardizinganalysis of patient images may be desirable if the data will be used inthe aggregate. Standardized analysis tools can control what steps aredone by whom or how steps are done, narrow ranges on steps, or limitsteps, and pre-process on server either outside of or within users'control.

Using data mining, data may be analyzed from different perspectives andsummarized into useful or relevant information. Data mining allows usersto analyze data from many different dimensions or angles, categorize thedata, and summarize the relationships identified. Clinical data miningmay be used, for example, to identify correlations or patterns amongfields in relational and/or other databases. The data mining system mayinclude a capability to compare data across modalities and/or datasources for a particular patient, for example.

In certain embodiments, system 109 may include a portal or interface(e.g., application programming interface or API) in which informationfor a patient may be accessed. Once a patient is identified, a userinterface is presented. The user interface includes patientdemographics, current order information, current patient information(e.g., medication, allergies, chief complaint, labs, etc.), historicalinformation (e.g., renal failure, family history, previous invasiveand/or non-invasive procedures, etc.), dynamic measurement analysis,and/or configurable normal values.

The data mining system provides a dynamic snapshot of vital measurementsand relevant findings across all studies in the medical data store 206for a particular patient. The data mining system supports access tomultiple data sources, cross-modality comparison, cross-data sourcecomparisons and the like. In some embodiments, the data mining systemallows data elements to be registered or subscribed so that a user,administrator and/or system setting may specify how to retrieve certaindata through a variety of communications protocols (e.g., SQL,extensible markup language (XML), etc.), what functions can be appliedto certain data, in which modality(ies) and/or data source(s) can acertain data element be found, whether data is enumerated and/or numericdata, etc.

The data mining in conjunction with the system 109 helps improveefficiency by reducing steps involving users, such as medical staff, toretrieve historical data and compare findings from previous procedures.This has previously been done manually, semi-manually or not at all. Thedata mining system retrieves, calculates and correlates dataautomatically. The data mining system may provide visual indicators ofdata relationships along with the data. In addition, the data miningsystem helps providing a more efficient workflow to compare and trenddata, which allows a physician or other healthcare practitioner to trackdisease progression and/or disease regression within the scope ofevidence-based medicine. FIG. 4C is a screenshot illustrating a GUIpresenting a result of data mining operations performed by a cloudserver according to one embodiment of the invention. In the examplerepresented in FIG. 4C, the current patient has a tumor, the diameter ofwhich is represented by the triangle on the graph. The other data on thegraph represent mined data of similar patients and/or similar tumors.The current patient's tumor diameter can be seen in context of the mineddata. Note that in this example, an alert is displayed that says thecurrent patient's tumor diameter is within the malignant range of data.Similar alerts would be generated depending on how the data isinterpreted by the advanced image processing system.

Some embodiments provide intelligent clinical data mining. Inintelligent data mining, data sets are generated based on relevant studyinformation. The data mining system may mine data across all studies fora particular patient which includes different modalities and potentiallymultiple data sources. The data mining system provides real-time (orsubstantially real-time) analysis of vital measurements and relevantfindings across all studies for a particular patient that helps improvea clinical ability to predict, diagnose and/or treat the patient.

In some embodiments, the data mining system provides interactivegraphing capabilities for mined data elements. For example, a user canselect different data points to be included in the graph, indicate atype of graph (e.g., line or bar), and select a size of the graph. Auser may select different function(s) to be applied to a specific dataelement such as change or difference, average, min, max, range, etc. Auser may utilize the data mining system to compare qualitative and/orquantitative data. The data mining system may be applicable to a widevariety of clinical areas and specialties, such as cardiology, diseaseprogression and regression, evidence-based medicine, radiology,mammography (e.g., track mass growth/reduction), interventionalneurology, radiology, cardiology (e.g., measure stenosis progression ofcarotid artery disease), hematology, oncology, etc.

In some embodiments the data mining system provides real-time orsubstantially real-time analysis that helps improve a clinical abilityto predict, diagnose and treat. Providing better tools and better accessto improved information leads to better decisions. Through trending andcomparing of clinical data, the data mining system has the ability togenerate graphs to give a user a visual representation of different dataelements.

For example, a cardiac physician may want to review findings fromprevious cardiac cases in order to compare and trend relevant data.Comparing and trending the data allows the physician to track diseaseprogression and/or disease regression. Pre-procedurally, the physicianis provided with an ability to be better prepared and informed ofrelevant clinical data that is pertinent to an upcoming procedure.Post-procedurally, the physician is provided with an ability to compareand trend the findings within the scope of evidence-based medicine totrack disease progression or regression and potentially recommend othertherapies. Access to the aggregated data can also be licensed to theuser separately from the use of the software itself.

Cloud-Based Medical Image Collaboration System

Referring back to FIG. 2, according to one embodiment, server 109further includes medical image collaboration system 208 capable ofhosting a medical image discussion and processing session amongstparticipants such as clients 202-203 discussing and processing medicalimages retrieved from medical data store 206 in a collaboration fashion.Each participant can participate (e.g., view and/or modify images) inthe discussion to a certain degree dependent upon his/her respectiveaccess privileges controlled by access control system 210. Differentparticipants may participate in different stages of a discussion sessionor a workflow process of the images. Dependent upon the access or userprivileges associated with their roles (e.g., doctors, insurance agents,patients, or third party data analysts or researchers), differentparticipants may be limited to access only a portion of the informationrelating to the images or processing tools without compromising theprivacy of the patients associated with the images. Participants mayalso communicate with each other in a collaborative manner, includingvia chat, instant messaging, voice or other means. This communicationmay be in either real time or recorded.

FIG. 5 is a block diagram illustrating a cloud-based image collaborationsystem according to one embodiment of the invention. Referring to FIG.5, system 500 includes a medical image collaboration system 208configured to host a discussion and processing forum in the cloudconcerning a medical image, which is accessible by clients 202-203 in acollaborated manner from anywhere in the world through the Internet. Inone embodiment, collaboration system 208 includes a collaboration module501 to coordinate communications and actions amongst clients 202-203discussing, viewing and/or manipulating host image and/or image data502. In response to an image manipulation or rendering command receivedfrom one of clients 202-203 on image 502, collaboration module 501 isconfigured to invoke image processing system 207 to render image 502according to the command. Collaboration module 501 is configured toupdate image and/or image data 502 based on the rendering resultreceived from image processing system 207.

In addition, collaboration module 501 generates, via image processingsystem 207, client images and/or image data 504-505 and transmits clientimages and/or image data 504-505 to clients 202-203, respectively.Client images and/or image data 504-505 may be viewed based on accessprivileges (e.g., part of access control list or ACL 503) of clients202-203. Certain information associated with host image and/or imagedata 502 may not be visible based on the access privileges of clients202-203. For example, if a user of client 202 is an auditor, clientimage 504 may not include patient information, based on the ACL as shownin FIG. 3B. However, if a user of client 203 is a physician, clientimage 505 may include the patient information based on the ACL as shownin FIG. 3B. Note that throughout this application, a client imagereferred to herein represents a client image file or files that mayinclude the actual image (e.g., same or similar to the host image) andother associated data such as metadata (e.g., DICOM tags or headers),description or notes of the image, and/or access control dataconfiguring client applications during processing the image, etc. In oneembodiment, GUIs or controls for invoking certain graphics renderingtools to manipulate host image and/or image data 502 via thecorresponding client images 504-505 may or may not be enabled oravailable at client applications 211-212 dependent upon the accessprivileges of users associated with clients 202-203, such as the oneshown in FIG. 3A. Furthermore, host image and/or image data 502 may bepart of a particular stage of a workflow process managed by workflowmanagement system 205. Some of clients 202-203 may participate in onestage and others may participate in another stage dependent upon thecorresponding ACL such as the one shown in FIG. 3C. In one embodiment,collaboration module 501 is configured to coordinate the imageprocessing stages amongst clients 202-203. When a first client hascompleted one stage, collaboration module 501 may send a notification toa second client such that the second client can take over the control ofthe image data and processing the image data of the next stage, etc.

FIGS. 6A-6D are screenshots illustrating certain GUIs of a medical imagecollaboration system according to certain embodiments of the invention.GUIs of FIGS. 6A-6D may be presented by a client application (e.g., thinclient) of various clients operated by various users. For example, FIG.6A represents a GUI page by a client application operated by a user thathas a high level of access privileges. Referring to FIG. 6A, in thisexample, the user can view most of the information presented by theimage collaboration system including most of the image processing tools605 and 608 that can be utilized to manipulate images and/or image data601-604, settings 606, workflow templates 607, image viewing tools 609,such as different orientations (anterior, head, posterior, right, foot,left), different views (axial, sagittal, coronal), different screenorientations, etc.], and patient information 610-613. FIG. 6B representsa GUI page that can be viewed by another user. Referring to FIG. 6B,this user may not have the necessary privileges to view the patientinformation. As a result, the patient information is not displayed. FIG.6C represent a GUI in which a user can only view the image without thecapability of manipulating the images. FIG. 6D represent a GUI in whicha user has a limited capability of manipulating the images and/or imagedata.

FIG. 7 is a flow diagram illustrating a method for processing medicalimage data according to one embodiment of the invention. Method 700 maybe performed by cloud server 109 of FIG. 1. Referring to FIG. 7, atblock 701, a cloud server receives over a network a request foraccessing three-dimensional (3D) medical image data from a first user.The cloud server provides image processing services to a variety ofusers over the network such as the Internet. At block 702, the cloudserver determines first user privileges of the first user for accessingthe 3D medical image data. The first user privileges may be related tothe 3D medical image data and may be configured by an owner of the 3Dmedical image data. Based on the first user privileges, the cloud serveris configured to limit the image processing tools available to the firstuser to process the 3D medical image data.

FIG. 8 is a flow diagram illustrating a method for processing medicalimage data in a collaboration environment according to anotherembodiment of the invention. Method 800 may be performed by a clientapplication such as client applications 211-212 of FIG. 5. Referring toFIG. 8, at block 801, image data is received at a client from an imageprocessing cloud server over a network, including information relatingto user privileges of a user of the client device. At block 802, theimage data is displayed via a client application running at the clientdevice. At block 803, one or more image processing interfaces (e.g.,buttons of a toolbar) of the client application is configured (e.g.,enabled/activated or disabled/deactivated) based on the user privileges.In response to a command received via one of the enabled imageprocessing interface, information representing a rendering command istransmitted to the image processing server over the network. In responseto updated image data received from the image processing server, theupdated image data is represented at the local client device. This typeof access control may occur with or without conferencing.

Conferencing/collaboration includes more than one user looking at and/orusing the advanced imaging software for a particular study, image orseries of images as services provided by a cloud at the same time or atdifferent times. This might include a physician consulting with apatient in another location, or a physician consulting with anotherphysician, or other users. One user may be using the software tomanipulate, measure, analyze images while other user(s) observe. Morethan one user may be actively using the software at the same time.Another example of collaboration is when more than one user iscontributing to a case or cases at different times. For example onephysician may perform certain steps or stages relating to patient imagedata, such as the bone removal step, and then another physician mightperform different steps at a later time, such as vesselidentification/labeling and/or measurement. Another user might reviewand validate certain previously performed steps, etc.

A cloud-based software system allows conferencing/collaboration to bedone on a level not possible before, using the techniques describedthroughout this application. Since the software and the data residecentrally (e.g., a single server, server farm or redundant servers), itis simply a matter of providing access to image data and access to theadvanced image processing software. The image data may relate to asubset of one procedure, one patient, or more than onepatient/procedure. The software and image data can be accessed fromanywhere and at any time, by anybody whom has been provided access,without extensive software are hardware installation. There are severalsituations where it would be desirable to have more than one user accessthe images and data relating to a procedure, patient or group ofpatients.

One such situation is when a user seeks a second opinion. For example, apatient, or physician, or insurer, may want to obtain a second opinionconcerning a procedure, patient or group of patients. By allowing morethan one physician access to the images and data of a case, a user(e.g., patient, physician, insurer, etc.) may request and obtain morethan one opinion concerning the case. The physicians may access the dataat different times, or at the same time. Each physician may want to viewthe steps and views that the other physician went through to come upwith his or her diagnosis/conclusion. So not only would two physiciansbe able to view the same case at either the same time or differenttimes, but one or more of the physicians, or other users, may be able toview a record of the steps that the other physician went through to comeup with his or her conclusion. In this scenario, users maysimultaneously or independently utilize the software. Different accessprivileges may be applied to different users.

Similar to the second opinion situation, a patient, physician or insurermay desire the opinion of an expert in a particular field. For example,a patient may receive a heart scan which reveals a rare condition. Thepatient, his/her physician and/or insurer may request the opinion of anexpert in the field of that rare condition. Similar to the secondopinion scenario, the users may view and manipulate, measure etc. theimages of the case simultaneously; user history can be tracked as in thesecond opinion scenario. In this scenario, users may simultaneously orindependently utilize the software.

In certain cases, for example in the case of a clinical trial, theUnited States Food and Drug Administration (FDA) may want to monitor theprogress and results. In this case, an individual or individuals at theFDA would be a user and may want to observe or monitor other users usingthe software to view or manipulate images and/or data. Therepresentative from the FDA may also be interested in looking atanonymous aggregated data. FDA users may only need to view the anonymousdata without the option of manipulating the data.

In another example, a representative from an insurer may want to monitorthe results from a patient or group of patients or doctor(s). In thiscase, an individual or individuals at the insurer would be a user andwould observe or monitor the results of other users using the software.The representative from the insurer may also be interested in looking atanonymous aggregated data. Insurance users may only need to view theanonymous data without the option of manipulating the data. They may usethe information for billing purposes or cost reductions/discounts.

Another potential use for collaborative advanced imaging software isduring a procedure, such as a surgical procedure. For example, a surgeonin a rural town may want the help of an expert at a major medicalcenter. Collaborative use of the software in the actual operating roomwould allow the surgeon to benefit from the guidance of the expert inreal time. The expert could also help plan the surgical procedurebeforehand using the advanced imaging software collaboratively. As withany of the embodiments, different access privileges may be applied todifferent users.

In another scenario, a medical center might outsource all or part of itsadvanced image processing operations to an outside company. Or it mightoutsource only certain types of cases. Or it might outsource certainsteps in cases which are more complex. The usage of the software can belicensed in a variety of licensing models. A patient may control theprocess—this opens up the opportunity for patients to have more controlover their care. For example, the patient might control a username andpassword for their case which they can then give to anyone, including asecond opinion doctor, etc. The username and password may be temporary,expiring after a configured time frame or a number of uses or otherparameter. Different users such as doctors or technicians, or expertscan do different steps or stages in image processing, for example, in aworkflow process.

Training is another scenario in which the conferencing and collaborationcan be utilized. In a training environment, there tends to be a largenumber of users. Some features of a testing environment include testing,lectures, certification and others. In a testing situation, severalusers will use the advanced image processing software to view andmanipulate or analyze image data on the same case or different cases,either simultaneously, or independently. Test scoring can be performedautomatically, or by an instructor viewing the result and process of thevarious students. The results may be quantitative or qualitative. In alecture situation, there may be a need to present cases to students in afairly controlled manner, by limiting what the students can see on anygiven case, or by magnifying certain aspects of a case for closerviewing and/or emphasis. In some situations, there may be a need to havethe same case presented in two windows on the student's computer so thatthe student can see what the instructor is doing in one window, but canalso use the advanced imaging software independently on the same case inthe other window. There may be a need to certify a user to do certaintypes of cases using the advanced image processing software or to usecertain aspects of the advanced image processing software. This wouldinvolve a fairly structured course with a test or tests which must bepassed in order for the user to be certified. This type of trainingcould be done live, or as an online course which is self-paced.

Traditionally, advanced image processing software has been usedprimarily by radiologists. This is largely because radiologists haveaccess to the workstation with the software installed at the medicalcenter. But other physicians and technicians in other specialties, suchas cardiology, orthopedics, dentistry, neurology, pathology, etc., wouldalso benefit from using this type of advanced image processing system.Since the cloud-based system effectively eliminates the need forexpensive and extensive hardware and software installation andmaintenance, access to advanced image processing software in the cloudbecomes possible for any type of physician or technician, whether or nothe/she is associated with a medical institution. An individual physicianin private practice can use the software at even its most advanced levelimmediately and without up-front costs and installation delays.

Since advanced image processing software is complex, training may berequired before a user is proficient using the software. However,different levels of software can be created for different user levels. A“dumbed down” version, which does not include the more complex tools,can be created for basic users, such as a primary care physician. Moreadvanced versions of the software can be created for more advanced userssuch as radiologists who have been trained to use the more advancedtools. Different specialties can also have different versions of thesoftware. For example, cardiologist may only need and want access to theadvanced image processing tools relating to the heart.

Cloud-Based Medical Data Anonymous Gateway Management

In order to truly use advanced image processing software in a cloudmodel, it is necessary to receive the slice image data from themodality, or scanner, onto a server in the cloud. Different types ofmedical image capturing devices include CT, MRI, PET, single photonemission computed tomography (SPECT), ultrasound, tomography, etc.Traditionally, the lack of efficient and effective methods for movinglarge volumes of image data has prevented the transition fromworkstation and server-based systems to a cloud-based system. In thepast, fewer scans were performed and there were fewer image slices perscan. The volume of data is increasing quickly creating the need for aneasy way of transferring significant numbers of large images efficientlyand without extensive setup.

Currently, most image data is transferred from a medical image capturingdevice to a PACS system. A PACS system generally exists at a medicalinstitution behind a firewall to protect the data. The existence of afirewall can make the transfer of files in either direction challengingbecause of security policies and rules. A virtual private network (VPN)has been used to transfer files, but installation of a VPN requiresextensive involvement from the hospital or medical centers IT departmentto make sure that all security policies are considered and that thesystem integrates seamlessly with their current system. For this reason,the setup of a VPN is time consuming, unpredictable, and sometimesimpossible or impractical.

According to some embodiments, a cloud-based medical image processingsystem includes a data gateway manager to automatically transfer medicaldata to/from data providers such as medical institutes. Such datagateway management may be performed based on a set of rules or policies,which may be configured by an administrator or authorized personnel. Inone embodiment, in response to updates of medical image data during animage discussion session or image processing operations performed at thecloud, the data gateway manager is configured to transmit over a network(e.g., Internet) the updated image data or the difference between theupdated image data and the original image data to a data provider thatprovided the original medical images. Similarly, the data gatewaymanager is configured to transmit any new images from the data provider.In addition, the data gateway manager may further transfer data amongstmultiple data providers that is associated with the same entity (e.g.,multiple facilities of a medical institute). Furthermore, thecloud-based system may automatically perform certain pre-processingoperations of the received image data using certain advanced imageprocessing resources provided by the cloud systems.

FIG. 9 is a block diagram illustrating a cloud-based image processingsystem according to another embodiment of the invention. Referring toFIG. 9, system 900 includes cloud 103 having one or more cloud servers109 to provide image processing services to a variety of clients forprocessing images stored in medical data store 206. Image data stored indata store 206 may be received over network 201 (e.g., Internet) from avariety of data sources such as data centers 101-102. Server 109 mayinclude some or all of the functionalities described above. Each of datacenters 101-102 includes a data store such as data stores 905-906 tostore or archive medical image data captured by a variety of imagecapturing devices, such as CT, MRI, PET, SPECT, ultrasound, tomography,etc. The data centers 101-102 may be associated with differentorganization entities such as medical institutes. Alternatively, datacenters 101-102 may be associated with different facilities of the sameorganization entity.

In one embodiment, each of data centers 101-102 includes a data gatewaymanager (also referred to as an uploader and/or downloader) such as datagateway managers 901-902 to communicate with cloud 103 to transfermedical data amongst data stores 905-906 and 206, respectively. Datagateway managers 901-902 can communicate with the cloud server accordingto a variety of communications protocols, such as hypertext transferprotocol secure (HTTPS) (e.g., HTTP with transport layer security/securesockets layer (TLS/SSL)), etc., using a variety of encryption and/orcompression techniques.

In one embodiment, for the purpose of illustration, when new image datais received from an image capturing device and stored in data store 905,data gateway manager 901 is configured to automatically transmit the newimage data to cloud 103 to be stored in data store 206. The new imagedata may be transmitted to cloud 103 and stored in a specific area ordirectory of data store 206 based on the configuration or profile setforth in a set of rules 903. Rules 903 may be configured by a user or anadministrator via an API such as a Web interface. Similarly, if new orupdated image data is received from a user and stored in data store 206,for example, during a Web conferencing session, data gateway manager 907may be configured to automatically transmit the new or updated imagedata to data center 101 according to a set of rules 903, which may alsobe configured by a user or administrator. According to some embodiments,data gateway managers 901-902 may also communicate with each other totransfer data stored in data stores 905-906, particularly, when datacenters 101-102 are associated with the same organization, or associatedin some other way, as in a clinical or research trial.

According to one embodiment, each of data gateway managers 901-902includes a data anonymizer such as anonymizers 909-910, prior totransmitting medical data to cloud 103 or amongst data centers 101-102,configured to anonymize certain information from the medical data, suchas patient information including, but is not limited to, patient name,address, social security number, credit card information, etc. Such ananonymization can be performed based on rules which are preconfigured,or configured at the time of data transfer and/or in an anonymizationconfiguration file (e.g., anonymization configuration files 911-912),which can be configured by a user or an administrator via anadministration user interface associated with the data gateway manager.Note that data gateway management may also be performed with other datasources 922 (e.g., image storage systems, CD-ROMS, computer hard drivesetc.) via the respective data gateway manager 921 and anonymizer 920.

According to some embodiments, the data gateway managers 901-902 allow auser (e.g., clinic, hospital, private practice, physician, insurer,etc.) to easily, and in some cases, automatically, upload image data(and/or other data) to server 109 and stored in medical data store 206in cloud 103. The data gateway manager can be configured using a Webbrowser interface, where the configuration may be stored as a set ofrules 903-904 or the rules can be determined at the time of datatransfer by a user. Certain internet ports such as ports 80 and 443 canbe used for such data transfers. Using these protocols allows the user(institution, hospital etc.) to use ports and protocols that in mostcases are already set up for secure transfer of data without setting upa separate VPN.

The Web interface allows a user to configure the file transfer using aWeb browser. This can be done each time image data is transferred to thecloud, or can be set up to automatically upload certain image data,cases or potions of cases based on rules. Some examples of rules can beconfigured based on modality, patient, dates, doctors, times, flags,clinical trial qualifications, multiple criteria, etc. Generally,patient identifying data needs to be removed from the image data beforethey are transferred or during transfer. This is referred to as‘anonymization.” This can be done in a number of ways and can also beautomated using rules such as based on birth date, upload date,institution, etc.

The anonymization can be done in a number of ways including, but notlimited to, blanking or masking out characters in the DICOM header,replacing characters in the DICOM header with non-identifyingcharacters, substitution, encryption, transformation, etc. Depending onthe anonymization methods, de-anonymization, or partialde-anonymization, may be possible. For example in a clinical trial, if apatient is experiencing unacceptable side effects, it would be desirableto de-anonymize their clinical trial data to determine whether thepatient was taking a placebo or a drug. The access control would benecessary so that only those users with certain privileges would beallowed to de-anonymize the data.

FIGS. 10A and 10B are screenshots illustrating examples of graphicaluser interfaces for configuring the gateway manager according to certainembodiments of the invention. For example, the GUIs as shown in FIGS.10A and 10B may be presented by client applications 211-212 of FIG. 2,which may be a client application or Web browser interface. The GUI forthe gateway manager may also be an application separate from clientapplications 211-212. Referring to FIG. 10A, in one embodiment, GUI 1000allows a user to configure the data files to be transferred to/from aserver such as server 109 of FIG. 2. A user can add one or more filesinto the GUI 1000 via link 1006 or alternatively, by drag-and-droppingone or more individual files and/or a folder of files into GUI 1000.Each file to be transmitted to/from the server is associated with anentry shown in GUI 1000. Each entry includes, but is not limited, statusfield indicating a file transfer status, time field 1002 indicating thetime when the file was added, progress indication 1003, source data path1004 from which the file is retrieved, and security indicator 1005indicating whether the transfer is conducted in a secure manner, etc.

A file can be manually selected, for example, by drag-and-dropping thefile into a specific folder. The file is then anonymized, compressed,and/or encrypted, and uploaded to the cloud, as shown in FIG. 12A.Alternatively, a router and/or gateway manager can be configured toautomatically upload to the cloud, with optional anonymization,compression, and/or encryption, image data captured by an imaging deviceand stored in a medical data store, as shown in FIG. 12B. Furthermore, arouter and/or gateway manager can also be configured to automaticallydownload from the cloud, with optional anonymization, compression,and/or encryption, image data and to store the image data in a medicaldata store, as shown in FIG. 12C. The protocol used for transferringdata may be a DICOM transmission protocol or other appropriate protocolor protocols. DICOM transmission protocol may use the following TCP andUDP port numbers: Port 104 is a well-known port for DICOM over TCP orUDP. Since port 104 is in the reserved subset, many operating systemsrequire special privileges to use it. Port 2761 is a registered port forDICOM using Integrated Secure Communication Layer (ISCL) over TCP orUDP. Port 2762 is a registered port for DICOM using Transport LayerSecurity (TLS) over TCP or UDP. Port 11112 is a registered port forDICOM using standard, open communication over TCP or UDP.

Once files associated with a study have been uploaded to the cloudserver, the user may be able to see a list which studies are in thataccount (not shown). The list identifies the status of the studiesand/or files, including whether it has been downloaded previously and/orwhether it has been read or changed. The user has the option ofautomatically or manually downloading any changed studies which have notbeen previously downloaded, or choosing which studies to download. Theuser also has the option of showing only studies that have been changedand not downloaded in the account or sort the list so that thenon-downloaded studies are listed at the top of the list. When filesand/or studies are specified for download, either automatically via link1013 or manually via link 1012, the files/studies can be downloaded to aspecific location in a local computer hard drive and/or thefiles/studies can be downloaded to a non-local computer hard drive.

According to one embodiment, a file whose transfer fails due to an errorcan be resent via link 1007. An alert can also be sent via email,displayed on a screen pop-up or added to a database for listing. Thesoftware associated with GUI 1000 may be automatically updated via link1010 or manually updated via link 2011. The destination or server toreceive the files can also be configured via link 1009. When link 1009is activated, GUI 1050 is presented as shown in FIG. 10B. Referring nowto FIG. 10B, in GUI 1050, a user can specify the username via field 1051and password in field 1052 to access the server specified via field1054. The user can also specify a group or directory to which the fileor files to be transferred via field 1053. The user can also specifywhether a secured connection is needed via field or checkbox 1055 and adata compression method is utilized via field or checkbox 1056.Compression can be either lossless or lossy. Further, the user canprovide an email address via field 1057 and the email system via field1058 to allow the system to send an email to the user regarding the datatransfer. The user can also specify whether the data will be anonymizedvia field or checkbox 1059, as well, as the data logging detailed levelvia field 1060. Furthermore, a user can specify by enabling checkbox1061 that the data is to be stored in an encrypted form. The user canalso provide a key via fields 1062-1063 to a recipient to decrypt thedata.

FIG. 11 is a screenshot illustrating examples of GUIs for configuringanonymous data gateway management according to certain embodiments ofthe invention. GUI 1100 can be presented by activating link 1008 of FIG.10A. Referring to FIG. 11, GUI 1100 allows a user to specify items to beanonymized, where each item is specified in one of the entries listed inGUI 1100. Each item is referenced by its DICOM tag 1101, name 1102,original value 1103 to be replaced, and replacement value 1104 toreplace the corresponding original value. In this example, a user canset the new value by clicking column 1104 and enter the new value. Ifthe DICOM tag is a date type, a date selector will be displayed to allowthe user to select the date. If the values allowed are predefined, adrop down list is displayed for selecting one of the predefined valuesor strings. If there is a mask defined for the tag, a masked edit GUI isdisplayed to allow the user to change the value according to thedisplayed mask. The user input maybe examined by the system based on thetype of the DICOM tag. If the information is incorrect, the user may beprompted to reenter the correct value. After all user inputs arecollected, a new anonymous template or configuration file is created andstored. Note that the GUIs as shown in FIGS. 10A-10B and 11 can bepresented and operated via a variety of user interactions such askeystrokes, clicking, touching, voice interactive commands, or acombination thereof. Also note that the formats or configurations of theGUIs in FIGS. 10A-10B and 11 are described for the purpose ofillustration only; other formats or layouts may also be utilized. TheGUI may be in the form of a browser or a phone or other mobile deviceapplication.

FIG. 13 is a flow diagram illustrating a method for anonymizing medicaldata according to another embodiment of the invention. Method 1300 maybe performed by any of data gateway managers 901-902, 921 of FIG. 9.Referring to FIG. 13, at block 1301, a local device (e.g., gatewaymanager/router/computer) receives a 3D medical image data captured by amedical imaging device. At block 1302, the 3D medical image data isanonymized including removing certain metadata associated with the 3Dmedical image data based on an anonymization template. At block 1303,the anonymized 3D medical image data is then automatically uploaded to acloud server, using a network connection established via an internetport of the local device.

Applications of Cloud-Based Services

The embodiments described above can be applied to a variety of medicalareas. For example, the techniques described above can be applied tovessel analysis (including Endovascular Aortic Repair (EVAR) andelectrophysiology (EP) planning). Such vessel analysis is performed forinterpretation of both coronary and general vessel analysis such ascarotid and renal arteries, in addition to aortic endograft andelectro-physiology planning. Tools provided as cloud services includeauto-centerline extraction, straightened view, diameter and lengthmeasurements, Curved Planar Reformation (CPR) and axial renderings, aswell as charting of the vessel diameter vs. distance and cross-sectionalviews. The vessel track tool provides a Maximum Intensity Projection(MIP) view in two orthogonal planes that travels along and rotates aboutthe vessel centerline for ease of navigation and deep interrogation.Plaque analysis tools provide detailed delineation of non luminalstructure such as soft plaque, calcified plaque and intra-mural lesions.

In addition, the techniques described above can be utilized in the areaof endovascular aortic repair. According to some embodiments, vascularanalysis tools provided as cloud services support definition of reporttemplates which captures measurements for endograft sizing. Multiplecenterlines can be extracted to allow for planning of EVAR procedureswith multiple access points. Diameters perpendicular to the vessel maybe measured along with distances along the two aorto-iliac paths. Customworkflow templates may be used to enable the major aortic endograftmanufactures' measurement specifications to be made as required forstent sizing. Sac segmentation and volume determination with a“clock-face” overlay to aid with documenting the orientation andlocation of branch vessels for fenestrated and branch device planning,may also be used. Reports containing required measurements and data maybe generated.

The techniques described above can also be applied in the left atriumanalysis mode, in which semi-automated left atrium segmentation of eachpulmonary vein ostium is supported with a single-click distance pairtool, provided as cloud services, for assessment of the major and minorvein diameter. Measurements are automatically detected and captured intothe integrated reporting system. These capabilities can be combined withother vessel analysis tools to provide a comprehensive and customized EPplanning workflow for ablation and lead approach planning.

The techniques described above can also be utilized in calcium scoring.Semi-automated identification of coronary calcium is supported withAgatston, volume and mineral mass algorithms being totaled and reportedon-screen. Results may be stored in an open-format database along withvarious other data relating to the patient and their cardiovascularhistory and risk factors. A customized report can be automaticallygenerated, as part of cloud services, based upon these data. Alsoincludes report generation as defined by the Society of CardiovascularComputed Tomography (SCCT) guidelines.

The techniques described above can also be utilized in a time-volumeanalysis (TVA), which may include fully-automated calculation of leftventricular volume, ejection fraction, myocardial volume (mass) and wallthickening from multi-phasic data. A fast and efficient workflowprovided as part of cloud services allows for easy verification oradjustment of levels and contours. The results are presented within theintegrated reporting function.

The techniques described above can also be utilized in the area ofsegmentation analysis and tracking (SAT), which includes supportsanalysis and characterization of masses and structures in various scans,including pulmonary CT examinations. Features include single-clicksegmentation of masses, manual editing tools to resolve segmentationissues, automatic reporting of dimensions and volume, graphical 3Ddisplay of selected regions, integrated automated reporting tool,support for follow-up comparisons including percent volume change anddoubling time, and support for review of sphericity filter results.

The techniques described above can also be utilized in the area offlythrough which may include features of automatic segmentation andcenterline extraction of the colon, with editing tools available toredefine these centerlines if necessary. 2D review includes side-by-sidesynchronized supine and prone data sets in either axial, coronal orsagittal views with representative synchronized endoluminal views. 3Dreview includes axial, coronal and sagittal MPR or MIP image displaywith large endoluminal view and an unfolded view that displays theentire colon. Coverage tracking is supported to ensure 100% coveragewith stepwise review of unviewed sections, one-click polypidentification, bookmark and merge findings, as well as a cube view forisolating a volume of interest and an integrated contextual reportingtool. Support is provided for use of sphericity filter results.

The techniques described above can also be utilized in the area oftime-dependent analysis (TDA), which provides assessment tools foranalyzing the time-dependent behavior of appropriate computerizedtomographic angiography (CTA) and/or MRI examinations, such as withincerebral perfusion studies. Features include support for loadingmultiple time-dependent series at the same time, and a proceduralworkflow for selecting input and output function and regions ofinterest. An integrated reporting tool is provided as well as theability to export the blood flow, blood volume and transit time maps toDICOM. The tools may also be used with time-dependent MR acquisitions tocalculate various time-dependent parameters.

The techniques described above can also be utilized in the area ofCTA-CT subtraction, which includes automatic registration of pre- andpost-contrast images, followed by subtraction or dense-voxel maskingtechnique which removes high-intensity structures (like bone andsurgical clips) from the CTA scan without increasing noise, and leavingcontrast-enhanced vascular structures intact.

The techniques described above can also be utilized in dental analysis,which provides a CPR tool which can be applied for review of dental CTscans, offering the ability to generate “panoramic” projections invarious planes and of various thicknesses, and cross-sectional MPR viewsat set increments along the defined curve plane.

The techniques described above can also be utilized in the area ofmulti-phase MR (basic, e.g. breast, prostate MR). Certain MRexaminations (for example, breast, prostate MR) involve a series ofimage acquisitions taken over a period of time, where certain structuresbecome enhanced over time relative to other structures. This modulefeatures the ability to subtract a pre-enhancement image from allpost-enhancement images to emphasize visualization of enhancingstructures (for example, vascular structures and other enhancingtissue). Time-dependent region-of-interest tools are provided to plottime-intensity graphs of a given region.

The techniques described above can also be utilized in parametricmapping (e.g. for multi-phase Breast MR), in which the parametricmapping module pre-calculates overlay maps where each pixel in an imageis color-coded depending on the time-dependent behavior of the pixelintensity. The techniques described above can also be utilized in thearea of SphereFinder (e.g. sphericity filter for lung and colon).SphereFinder pre-processes datasets as soon as they are received andapplies filters to detect sphere-like structures. This is often usedwith lung or colon CT scans to identify potential areas of interest. Thetechniques described can also be utilized in fusion for CT/MR/PET/SPECT.Any two CT, PET, MR or SPECT series, or any two-series combination canbe overlaid with one assigned a semi-transparent color coding and theother shown in grayscale and volume rendering for anatomical reference.Automatic registration is provided and subtraction to a temporary seriesor to a saved, third series is possible.

The techniques described above can also be utilized in the area ofLobular Decomposition. Lobular Decomposition is an analysis andsegmentation tool that is designed with anatomical structures in mind.For any structure or organ region which is intertwined with a tree-likestructure (such as an arterial and/or venous tree), the LobularDecomposition tool allows the user to select the volume of interest, aswell as the trees related to it, and to partition the volume into lobesor territories which are most proximal to the tree or any specificsub-branch thereof. This generic and flexible tool has potentialresearch applications in analysis of the liver, lung, heart and variousother organs and pathological structures.

The techniques described above can also be utilized in the area ofVolumetric Histogram. Volumetric Histogram supports analysis of a givenvolume of interest based on partition of the constituent voxels intopopulations of different intensity or density ranges. This can be used,for example, to support research into disease processes such as cancer(where it is desirable to analyze the composition of tumors, in anattempt to understand the balance between active tumor, necrotic tissue,and edema), or emphysema (where the population of low-attenuation voxelsin a lung CT examination may be a meaningful indicator of earlydisease).

The techniques described above can also be utilized in the area ofMotion Analytics. Motion Analytics provides a powerful 2D representationof a 4D process, for more effective communication of findings wheninteractive 3D or 4D display is not available. Any dynamic volumeacquisition, such as a beating heart, can be subjected to the MotionAnalysis, to generate a color-coded “trail” of outlines of keyboundaries, throughout the dynamic sequence, allowing a single 2D frameto capture and illustrate the motion, in a manner that can be readilyreported in literature. The uniformity of the color pattern, or lackthereof, reflects the extent to which motion is harmonic, providingimmediate visual feedback from a single image.

The techniques described above can also be utilized to support otherareas such as Multi-KV, enhanced multi-modality, findings workflow, andiGENTLE available from TeraRecon. Multi-KV: Support for Dual Energy andSpectral Imaging provides support for established applications of dualenergy or spectral imaging CT data, such as removal of bone or contrast,as well as toolkits to support research and investigation of newapplications of such imaging techniques. Enhanced multi-modality supportis offered, including support for PET/MR fusion, and improvedapplications for MR such as time-intensity analysis and parametricmapping tools, which may be applied in the study of perfusioncharacteristics of normal or cancerous tissue.

Findings Workflow supports progressive analysis of serial acquisitions,for the same patient. Each finding can be tracked across multipleexaminations, in a table that is maintained indefinitely in theiNtuition system's database, without requiring the prior scans to remainpresent on the system. Measurement data and key images are captured andretained, allowing new scans to be placed in context with prior results,and reports to be produced at any time. Support for RECIST 1.1 isincluded although the tool may readily be used for analysis of variousprogressive conditions, not only those related to oncology. Export usingthe AIM (Annotation and Image Markup) XML Schema is supported.

iGENTLE ensures that iNtuition's powerful suite of segmentation,centerline, and metadata extraction tools continue to work effectively,even with noisy scans characterized by low-dose acquisitions. Metadataare extracted from enhanced copies of the original scan, and thenapplied back onto the original, unmodified data, to improve performanceof 3D tools without denying access to the original scan data.

Example of Data Processing System

FIG. 14 is a block diagram of a data processing system, which may beused with one embodiment of the invention. For example, the system 1400may be used as part of a server or a client as shown in FIG. 1. Notethat while FIG. 14 illustrates various components of a computer system,it is not intended to represent any particular architecture or manner ofinterconnecting the components; as such details are not germane to thepresent invention. It will also be appreciated that network computers,handheld computers, cell phones and other data processing systems whichhave fewer components or perhaps more components may also be used withthe present invention.

As shown in FIG. 14, the computer system 1400, which is a form of a dataprocessing system, includes a bus or interconnect 1402 which is coupledto one or more microprocessors 1403 and a ROM 1407, a volatile RAM 1405,and a non-volatile memory 1406. The microprocessor 1403 is coupled tocache memory 1404. The bus 1402 interconnects these various componentstogether and also interconnects these components 1403, 1407, 1405, and1406 to a display controller and display device 1408, as well as toinput/output (I/O) devices 1410, which may be mice, keyboards, modems,network interfaces, printers, and other devices which are well-known inthe art.

Typically, the input/output devices 1410 are coupled to the systemthrough input/output controllers 1409. The volatile RAM 1405 istypically implemented as dynamic RAM (DRAM) which requires powercontinuously in order to refresh or maintain the data in the memory. Thenon-volatile memory 1406 is typically a magnetic hard drive, a magneticoptical drive, an optical drive, or a DVD RAM or other type of memorysystem which maintains data even after power is removed from the system.Typically, the non-volatile memory will also be a random access memory,although this is not required.

While FIG. 14 shows that the non-volatile memory is a local devicecoupled directly to the rest of the components in the data processingsystem, the present invention may utilize a non-volatile memory which isremote from the system; such as, a network storage device which iscoupled to the data processing system through a network interface suchas a modem or Ethernet interface. The bus 1402 may include one or morebuses connected to each other through various bridges, controllers,and/or adapters, as is well-known in the art. In one embodiment, the I/Ocontroller 1409 includes a USB (Universal Serial Bus) adapter forcontrolling USB peripherals. Alternatively, I/O controller 1409 mayinclude an IEEE-1394 adapter, also known as FireWire adapter, forcontrolling FireWire devices.

Some portions of the preceding detailed descriptions have been presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory. These algorithmic descriptions andrepresentations are the ways used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as those set forth in the claims below, refer to the actionand processes of a computer system, or similar electronic computingdevice, that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The techniques shown in the figures can be implemented using code anddata stored and executed on one or more electronic devices. Suchelectronic devices store and communicate (internally and/or with otherelectronic devices over a network) code and data using computer-readablemedia, such as non-transitory computer-readable storage media (e.g.,magnetic disks; optical disks; random access memory; read only memory;flash memory devices; phase-change memory) and transitorycomputer-readable transmission media (e.g., electrical, optical,acoustical or other form of propagated signals—such as carrier waves,infrared signals, digital signals).

The processes or methods depicted in the preceding figures may beperformed by processing logic that comprises hardware (e.g. circuitry,dedicated logic, etc.), firmware, software (e.g., embodied on anon-transitory computer readable medium), or a combination of both.Although the processes or methods are described above in terms of somesequential operations, it should be appreciated that some of theoperations described may be performed in a different order. Moreover,some operations may be performed in parallel rather than sequentially.

In the foregoing specification, embodiments of the invention have beendescribed with reference to specific exemplary embodiments thereof. Itwill be evident that various modifications may be made thereto withoutdeparting from the broader spirit and scope of the invention as setforth in the following claims. The specification and drawings are,accordingly, to be regarded in an illustrative sense rather than arestrictive sense.

What is claimed is:
 1. A computer-implemented method for providingcloud-based image processing services, the method comprising: receiving,at a cloud server over a network, a request for accessing medical imagedata from an application running at a client device used by a user, thecloud server providing image processing services to a plurality of usersin a plurality of image processing steps provided by the cloud server,resulting in a plurality of image views; determining user privileges ofthe user for accessing the medical image data, wherein the medical imagedata was captured by a medical imaging device and stored in a storageassociated with the cloud server; in response to receiving a commandfrom the application of the client device, the command having aselection of one or more of image views available to the user, providingby the cloud server the medical image data based on the selected one ormore image views provided by the cloud server; tracking userinteractions of the user with the medical image data via the selectedimage views, including tracking how long in time the user has spent on aparticular image view; storing the tracked user interactions in apersistent storage, wherein the persistent storage is configured totrack and store user interactions of a plurality of users at the cloudserver; and performing an analysis on the tracked user interactionsstored in the persistent storage to determine an overall usage trend ofthe image views.
 2. The method of claim 1, wherein tracking userinteraction comprises recording user activities of the user with themedical image data in view of the user privileges.
 3. The method ofclaim 2, wherein tracking user interaction further comprises recordingwhich of a plurality of medical image data processing steps the user hasinteracted with.
 4. The method of claim 3, further comprisingdetermining how much time the user has spent on a particular medicalimage processing step.
 5. The method of claim 2, wherein tracking userinteraction further comprises recording a user type of the user, whichis one of an expert user, a regular user, a technician, and a physician.6. The method of claim 2, wherein tracking user interaction furthercomprises tracking billing information of the user associated with usageof the image processing steps.
 7. The method of claim 1, wherein themedical image data is part of an electronic health record (EHR) of apatient received from an EHR provider over the network.
 8. The method ofclaim 7, wherein the application running at a client device is medicalclient software integrated with EHR client software, and wherein themedical client software concurrently accesses the EHR record at an EHRserver associated with the EHT provider without having to specificallylaunch a separate application.
 9. The method of claim 8, wherein the EHRrecord and the medical image data are identified by a patient identifier(ID) identifying the patient at the EHR server and the cloud server,respectively.
 10. A non-transitory machine-readable medium havingexecutable instructions stored therein, which when executed by aprocessor, cause the processor to perform operations for providingcloud-based image processing services, the operations comprising:receiving, at a cloud server over a network, a request for accessingmedical image data from an application running at a client device usedby a user, the cloud server providing image processing services to aplurality of users in a plurality of image processing steps provided bythe cloud server, resulting in a plurality of image views; determininguser privileges of the user for accessing the medical image data,wherein the medical image data was captured by a medical imaging deviceand stored in a storage associated with the cloud server; in response toreceiving a command from the application of the client device, thecommand having a selection of one or more of image views available tothe user, providing by the cloud server the medical image data based onthe selected one or more image views provided by the cloud server;tracking user interactions of the user with the medical image data viathe selected image views, including tracking how long in time the userhas spent on a particular image view; storing the tracked userinteractions in a persistent storage, wherein the persistent storage isconfigured to track and store user interactions of a plurality of usersat the cloud server; and performing an analysis on the tracked userinteractions stored in the persistent storage to determine an overallusage trend of the image views.
 11. The non-transitory machine-readablemedium of claim 10, wherein tracking user interaction comprisesrecording user activities of the user with the medical image data inview of the user privileges.
 12. The non-transitory machine-readablemedium of claim 11, wherein tracking user interaction further comprisesrecording which of a plurality of medical image data processing stepsthe user has interacted with.
 13. The non-transitory machine-readablemedium of claim 12, wherein the operations further comprise determininghow much time the user has spent on a particular medical imageprocessing step.
 14. The non-transitory machine-readable medium of claim11, wherein tracking user interaction further comprises recording a usertype of the user, which is one of an expert user, a regular user, atechnician, and a physician.
 15. The non-transitory machine-readablemedium of claim 11, wherein tracking user interaction further comprisestracking billing information of the user associated with usage of theimage processing steps.
 16. The non-transitory machine-readable mediumof claim 10, wherein the medical image data is part of an electronichealth record (EHR) of a patient received from an EHR provider over thenetwork.
 17. The non-transitory machine-readable medium of claim 16,wherein the application running at a client device is medical clientsoftware integrated with EHR client software, and wherein the medicalclient software concurrently accesses the EHR record at an EHR serverassociated with the EHT provider without having to specifically launch aseparate application.
 18. The non-transitory machine-readable medium ofclaim 17, wherein the EHR record and the medical image data areidentified by a patient identifier (ID) identifying the patient at theEHR server and the cloud server, respectively.
 19. A data processingsystem, comprising: a processor; and a memory coupled to the processorto store instructions, which when executed from the memory, cause theprocessor to perform operations, the operations including receiving, ata cloud server over a network, a request for accessing medical imagedata from an application running at a client device used by a user, thecloud server providing image processing services to a plurality of usersin a plurality of image processing steps provided by the cloud server,resulting in a plurality of image views, determining user privileges ofthe user for accessing the medical image data, wherein the medical imagedata was captured by a medical imaging device and stored in a storageassociated with the cloud server, in response to receiving a commandfrom the application of the client device, the command having aselection of one or more of image views available to the user, providingby the cloud server the medical image data based on the selected one ormore image views provided by the cloud server, tracking userinteractions of the user with the medical image data via the selectedimage views, including tracking how long in time the user has spent on aparticular image view, storing the tracked user interactions in apersistent storage, wherein the persistent storage is configured totrack and store user interactions of a plurality of users at the cloudserver, and performing an analysis on the tracked user interactionsstored in the persistent storage to determine an overall usage trend ofthe image views.
 20. The system of claim 19, wherein tracking userinteraction comprises recording user activities of the user with themedical image data in view of the user privileges.
 21. The system ofclaim 20, wherein tracking user interaction further comprises recordingwhich of a plurality of medical image data processing steps the user hasinteracted with.
 22. The system of claim 21, wherein the operationsfurther comprise determining how much time the user has spent on aparticular medical image processing step.